Provably safe systems: the only path to controllable AGI

We describe a path to humanity safely thriving with powerful Artificial General Intelligences (AGIs) by building them to provably satisfy human-specified requirements. We argue that this will soon be technically feasible using advanced AI for formal verification and mechanistic interpretability. We further argue that it is the only path which guarantees safe controlled AGI. We end with a list of challenge problems whose solution would contribute to this positive outcome and invite readers to join in this work.Comment: 17 page

Autonomous technology and the greater human good

<p>Military and economic pressures are driving the rapid development of autonomous systems. We show that these systems are likely to behave in anti-social and harmful ways unless they are very carefully designed. Designers will be motivated to create systems that act approximately rationally and rational systems exhibit universal drives towards self-protection, resource acquisition, replication and efficiency. The current computing infrastructure would be vulnerable to unconstrained systems with these drives. We describe the use of formal methods to create provably safe but limited autonomous systems. We then discuss harmful systems and how to stop them. We conclude with a description of the ‘Safe-AI Scaffolding Strategy’ for creating powerful safe systems with a high confidence of safety at each stage of development.</p

The TESS-Keck Survey. VIII. Confirmation of a Transiting Giant Planet on an Eccentric 261 Day Orbit with the Automated Planet Finder Telescope

We report the discovery of TOI-2180 b, a 2.8 $M_{\rm J}$ giant planet orbiting a slightly evolved G5 host star. This planet transited only once in Cycle 2 of the primary Transiting Exoplanet Survey Satellite (TESS) mission. Citizen scientists identified the 24 hr single-transit event shortly after the data were released, allowing a Doppler monitoring campaign with the Automated Planet Finder telescope at Lick Observatory to begin promptly. The radial velocity observations refined the orbital period of TOI-2180 b to be 260.8$\pm$0.6 days, revealed an orbital eccentricity of 0.368$\pm$0.007, and discovered long-term acceleration from a more distant massive companion. We conducted ground-based photometry from 14 sites spread around the globe in an attempt to detect another transit. Although we did not make a clear transit detection, the nondetections improved the precision of the orbital period. We predict that TESS will likely detect another transit of TOI-2180 b in Sector 48 of its extended mission. We use giant planet structure models to retrieve the bulk heavy-element content of TOI-2180 b. When considered alongside other giant planets with orbital periods over 100 days, we find tentative evidence that the correlation between planet mass and metal enrichment relative to stellar is dependent on orbital properties. Single-transit discoveries like TOI-2180 b highlight the exciting potential of the TESS mission to find planets with long orbital periods and low irradiation fluxes despite the selection biases associated with the transit method.Comment: Published in A